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ABSTRACT 


The  need  is  established  for  using  Subject  Matter  Kxperts  as  authors 
of  materials  for  technical  training  for  OJT.  The  activities  and  major 
findings  as  a result  of  authoring  operations  on-site  are  reviewed.  The 
training  system  requirements  established  are  described,  and  the  ra- 
tionale for  using  a procedural  approach  to  authoring  and  task -related 
models  for  lesson  designs  is  given.  The  necessity  from  a manager's 
view  for  OJT  to  improve  work  performance  and  for  efficient  use  of  ex- 
perts as  authors  is  emphasized.  Examples  of  development  of  two  task- 
related  lessons  are  presented,  and  the  plan  for  a test  of  lesson  products 
is  discussed  briefly. 
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I.  INTRODUCTION 

A.  Problem  Statement 

The  goal  of  this  project  is  to  design,  deploy,  field  test,  and  evaluate  a system  to  enable 
local,  qualified  personnel  to  prepare  individualized  instructional  material  for  on-the-job  train- 
ing (OJT)  by  machine  delivery.  OJT  is  the  only  appropriate  type  of  training  for  most  opera- 
tional, maintenance,  and  repair  tasks  because  these  tasks  are  unique  to  particular  pieces  of 
equipment  and  to  the  environment  in  which  the  equipment  is  used. 

Currently,  OJT  is  conducted  by  qualified  operators  and  technicians  who  instruct  and  super- 
vise novice  personnel,  most  frequently  on  a one-to-one  basis.  These  same  operators  and  tech- 
nicians are  also  responsible  for  equipment  maintenance,  repair,  readiness,  and  operations. 

As  a consequence,  training  is  often  unavailable  and,  at  best,  inefficient.  Capturing  the  exper- 
tise of  these  technicians  and  operators  in  suitable  form  for  automated  delivery  could  provide 
continuous  availability  of  training,  decrease  the  demand  for  expert  personnel  to  conduct  one-to- 
one  instruction,  further  standardize  training,  and  increase  the  overall  effectiveness  of  OJT. 

While  the  feasibility  of  machine  delivery  of  instruction  at  the  work  site  has  been  demon- 
strated with  the  Lincoln  Terminal  System  (LTS),  the  acquisition  of  appropriate  instructional  ma- 
terial remains  a challenge.  Professional  technical  instructors  or  writers  are  not  suited  to  the 
task  of  generating  such  material  because  they  seldom  have  the  extensive  local  operational  expe- 
rience and  up-to-date  technical  knowledge  that  are  required  to  design  materials  with  the  neces- 
sary precision  and  relevance  of  content.  Operational  personnel  tend  not  to  have  tlie  requisite 
authoring  skills.  Procedures  and  facilities  developed  under  this  project  will  compensate  for 
this  lack  of  authoring  experience,  and  will  enable  locally  qualified  operational  experts  to  prepare 
materials  that  will  meet  the  needs  of  OJT. 

B.  Summary  of  Project  Activities 

The  first  phase  of  activities  of  the  project  was  concerned  with  the  development  of  two  model 
lessons  that  seemed  especially  appropriate  to  technical  training  in  OJT,  the  target  area  of  appli- 
cation. Procedures  to  author  each  of  these  were  developed.  The  procedure  for  authors  on  pre- 
paring a Pure  Procedure  was  based  on  earlier  experience  in  the  preparation  of  materials  for 
the  AN/TRC-97A  under  prior  contract  with  the  Air  Force  Communications  Service  (AFCS).  The 
other  lesson  type,  TPI  (Task  I’rocedure  and  Interpretation),  is  a new  product  developed  under 
this  contract  and  is  uniquely  suited  for  delivery  of  technical  training  by  means  of  a computer- 
based  system  at  the  work  site.  These  developments  were  reviewed  in  the  previous  Semiannual 
Technical  Summary. 

Operations  on-site  during  the  past  six  months  have  been  managed  by  a resident  Lincoln  per- 
son, acting  as  the  LTS  training  coordinator  and  supporting  Air  Fore,  technicians  in  the  author- 
ing of  lessons.  These  efforts  have  proven  very  productive  in  the  sense  that  observations  of  the 
authoring  process  led  to  a major  revision  of  the  procedures.  The  first  few  authors  read  through 
the  procedures  but  tended  to  ignore  them  in  preparing  the  lesson;  as  a result,  the  products  con- 
tained much  explanatory  and  little  task-related  material.  Control  of  the  authoring  process  was 
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achieved  by  dividing  the  procedure  into  three  steps:  preparation  of  an  outline,  of  a frame  flow- 
chart, and  of  the  First  Draft  in  notebook  form.  The  product  of  each  step  was  reviewed  by  tiie 
training  coordinator  before  proceeding.  At  first,  it  was  necessary  for  the  training  coordinator 
to  participate  in  the  authoring  process  because  of  deficiencies  in  the  procedures.  As  commu- 
nication with  the  author  improved,  the  monitoring  was  reduced  to  interaction  on  the  product  at 
the  completion  of  each  step.  (Typically,  after  the  author  makes  corrections  in  the  Final  Draft 
notebook,  the  lesson  is  taken  over  by  the  training  system  and  is  assigned  to  other  personnel  for 
artwork,  logic  preparation,  etc.  Capturing  the  expertise  of  the  SME  (Subject  Matter  Expert)  is 
complete  at  this  point  except  for  a review  and  checkout  of  the  final  form  before  commitment  to 
fiche.)  Several  of  the  lesson  projects  were  completed,  reviewed  by  peer  SMEs,  and  tried  out 
successfully  on  novice  personnel.  None  of  the  completed  lessons,  however,  were  prepared 
under  conditions  such  that  a formal  test  of  their  effectiveness  would  constitute  a valid  test  of  ttie 
authoring  procedure.  In  addition  to  revision  of  the  authoring  procedures,  a characterization  of 
the  role  of  the  SME  as  author  was  developed.  It  was  based  on  this  early  experience,  as  well  as 
on  authoring  efforts  by  two  Air  Force  personnel  assigned  to  do  their  work  at  Lincoln  Laboratory. 
The  result  of  analysis  of  the  SME's  authoring  task  is  presented  in  Sec.  II. 

A major  revelation  was  the  severe  difficulty  of  conducting  material-preparation  activities 
at  an  operational  site.  Three  facts  dominate:  (1)  an  SME  who  is  qualified  to  author  is,  by  def- 
inition, qualified  to  conduct  operations;  (2)  the  supply  of  qualified  personnel  is  chronically  short; 
and  (3)  training  always  has  second  priority  to  operational  missions.  Thus,  a number  of  lessons 
started  by  authors  in  this  project  were  left  incomplete  because  of  the  reassignment  of  personnel 
to  more-pressing  duties  on-site,  in  the  field,  and  overseas.  The  usual  one-on-one  OJT,  of 
course,  also  suffers  under  conditions  such  as  these,  proof  of  the  need  for  a better  system.  It 
has  been  suggested  as  an  alternative  that  the  training  materials  be  obtained  from  an  outside 
source.  However,  qualification  requires  task  and  equipment-related  knowledge,  in  addition  to 
general  technical  knowledge  obtained  through  recent  work  experiences.  Prior  experience  may 
no  longer  be  valid  because  this  kind  of  factual  information  is  easily  forgotten  and  the  facts  re- 
membered often  rapidly  become  outdated.  There  is  a risk  that  training  materials  prepared  by 
a lesser  expert  will  be  partially  irrelevant.  Much  worse,  they  may  actually  have  an  adverse 
effect  on  work  performance.  The  only  realistic  way  to  introduce  automated  delivery  to  OJT  in 
most  cases  is  for  managers  to  dedicate  a fraction  of  truly  qualified  SME  talent  to  materials 
preparation  in  critical  areas.  This  policy  will  assure  that  the  materials  are  effective  and  that 
all  qualified  personnel  can  rely  on  the  materials  to  relieve  them  of  training  duties  in  the  long 
run.  This  and  other  system  requirements  are  reviewed  in  Sec.  II.  It  also  seems  essential  that 
the  SME  work  away  from  the  operational  setting.  Because  of  the  difficulty  of  conducting  uninter- 
rupted authoring  activity  on-site,  two  authors  were  brought  to  Lincoln  Laboratory  to  obtain  ex- 
amples of  a valid  authoring  process.  This  effort  proved  entirely  successful;  it  is  described  in 
detail  in  Sec.  III. 

The  present  site,  because  of  the  other  demands  on  personnel,  is  also  not  satisfactory  for 
c onducting  extensive  tests  of  lesson  products.  Test  operations  have  been  initiated  at  the  2nd  CCC. 
at  Patrick  AFB  to  add  to  our  source  of  authors  and  test  subjects.  Substantial  efforts  have 
gone  into  preparation  of  hardware  and  software  to  support  field  operations.  Two  LTS-5  delivery 
units  have  been  completed,  and  one  delivered  to  each  site.  A duplex  software  system  has  been 
developed  that  continues  to  support  the  LTS-3S  materials,  the  AN/TRC-97A  Procedures  cur- 
rently in  use  as  familiarization  training  at  both  sites.  Each  unit  also  supports  lessons  that 
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incorporate  new  features  of  the  LTS-5  that  improve  the  versatility  of  instructional  delivery  and 
permit  recording  of  training  data  on-site.  Computer-based  experimental  author  support  facil- 
ities have  been  developed  and  are  in  use  at  Lincoln  Laboratory;  deployment  to  the  field  has  not 
been  feasible  because  of  the  multiplicity  of  sites  where  authoring  activity  is  planned.  Moreover, 
operations  such  as  logic  generation  and  data  dumps  have  been  reduced  to  elementary  procedures. 
Specific  needs  for  this  kind  of  support  can  readily  be  specified  by  the  author  and  executed  effec- 
tively by  support  personnel.  Authors  are  given  a chance  to  review  and  check  out  their  lessons 
on  an  I TS  unit  when  they  first  appear  on  fiche,  and  corrected  fiche  are  issued.  In  view  of  the 
scarcity  of  the  SME  as  a resource,  provision  of  these  kinds  of  support  services  for  authors  is 
probably  the  efficient  way  to  proceed  even  in  the  long  run. 

II.  SYSTEM  REQITREMENTS 

As  a result  of  the  activities  at  the  field  site,  both  general  requirements  for  OJT  and,  more 
specifically,  the  role  of  the  SME  as  author  have  been  clarified.  The  need  for  efficient  use  of 
the  SME  as  author  is  established,  and  these  findings  are  summarized  briefly  here  to  amplify 
other  system  requirements. 

A.  The  OJT  System 

Training  in  the  OJT  environment  is  carried  on  as  part  of  the  work  management  system. 

The  overriding  consideration  is  preparation  of  personnel  to  perform  work;  and  from  the  work 
manager's  viewpoint,  the  only  valid  criterion  for  evaluation  of  training  is  its  effect  on  work 
pe  rformance. 

Although  there  is  an  acute  need  for  training,  personnel  and  equipment  tend  to  be  dedicated 
to  unit  operations  other  than  training.  The  experience  at  the  5th  CCG  reinforces  the  necessity 
for  short-term  and  efficient  use  of  SMEs  as  the  source  of  materials  because  they  are  a scarce 
resource.  In  this  regard,  requirements  for  OJT  differ  from  those  for  traditional  schooling,  and 
the  SME  as  author  must  be  accommodated  to  achieve  a workable  system. 

Largely  as  a consequence  of  the  policy  of  continuous  reassignment  of  personnel,  there  is  a 
continual  influx  of  novices  to  receive  primary  OJT.  There  are  also  experienced  personnel  to  be 
cross-trained  who  may  be  handled  two  ways:  (1)  by  providing  broad  understanding  of  equipment 
and  operations  in  the  primary  OJT,  or  (2)  by  providing  additional  training  on  demand.  The  lat- 
ter strategy  seems  the  only  feasible  one  because  of  the  increasing  diversity  of  equipments  and 
systems  that  a technician  must  deal  with  in  his  career.  Specifically,  there  is  a need  for  train- 
ing materials  designed  to  accommodate  both  novice  and  experienced  personnel. 

OJT  must  have  a direct  benefit  in  terms  of  improvement  of  work  performance.  Rehearsal 
of  specific  work  procedures,  although  necessary,  is  not  always  sufficient.  The  level  of  training 
must  often  be  rather  sophisticated  to  apply  to  complex  technology.  In  areas  such  as  maintenance 
training,  there  is  a need  to  impart  cognitive  skills  of  judgment  such  as  decision-making  and 
problem-solving  at  moderate  levels  of  complexity.  That  is,  the  technician  must  have  an  appro- 
priate level  of  skill  knowledge  to  deal  with  unforeseen  problems. 

Another  major  requirement  of  the  training  system  is  flexibility  to  adapt  to  change  of  cir- 
cumstances — to  both  novice  and  experienced  personnel,  to  new  equipment,  to  new  operational 
doctrine,  and  to  new  missions.  The  need  for  adaptability  rules  out  developing  a large  stabilized 
system  for  OJT.  It  seems  more  profitable  to  regard  training  as  a branch  of  the  work  manage- 
ment system  constantly  under  development.  Within  the  work  system,  the  role  of  SME  as  author 


is  an  extension  of  his  role  as  operator  or  technician.  Specifically,  authoring  is  an  alternate 
means  for  providing  live  one-on-one  training. 

In  summary.  OJT  serves  to  improve  work  performance.  SMEs  must  be  used  extensively  in 
lesson  preparation,  but  are  a scarce  resource.  Trainees  have  a variety  of  backgrounds  to  be 
accommodated.  There  is  a need  for  training  in  cognitive  skills  beyond  memorization  of  terms 
and  work  sequences.  And,  flexibility  to  adapt  to  changing  system  and  work  conditions  is 
essential. 

B.  The  SMK  as  Author  in  OJT 

The  typical  candidate  for  authoring  is  an  expert  within  the  work  system  with  a repertoire 
of  technical  and  operational  skills,  from  the  work  manager's  viewpoint,  the  goal  is  to  use  this 
expertise  in  a different  wav,  capturing  it  in  training  materials  for  delivery  on  a computer-based, 
individualized  training  unit.  The  primary  duty  for  authors  of  this  kind  is  nonetheless  to  perform 
work,  and  the  best  of  them  are  often  a scarce  resource.  Therefore,  they  can  be  assigned  to 
lesson  preparation  only  for  brief  intervals.  To  expedite  lesson  preparation,  explicit  procedures 
have  been  developed.  Qualitative  observations  of  authoring  by  procedure  to  date  indicate  that  it 
is  a feasible  and  practical  approach. 

At  first,  it  is  not  obvious  that  the  SME  can  produce  materials  which  impart  requisite  cog- 
nitive skills  such  as  judgment  and  reasoning.  Instruction  at  this  level  has  long  been  considered 
to  depend  on  instructional  expertise.  In  fact,  some  SMEs  are  rather  unskilled  in  this  regard, 
as  exhibited  in  their  attempts  to  conduct  one-on-one  OJT.  In  the  information-test-correction 
cycle  of  computer-based  instruction,  the  information  is  the  most  difficult  part  for  the  SME  to 
provide.  In  this  area,  the  author  is  helped  by  presentation  of  a few  practical  rules  and  sugges- 
tions and  bv  examples  of  displays.  The  test-and-correction  phase  on  the  other  hand  is  seldom 
difficult  for  the  author  to  handle.  Much  of  the  OJT  involves  practical  judgment  which,  although 
at  times  a bit  complicated,  is  rarely  esoteric.  In  this  context,  the  primary  function  of  a test 
is  not  the  cognitive  one  of  testing  comprehension  but  a motivational  one.  If  the  trainee  expects 
to  be  tested  on  every  explanation  offered,  he  or  she  has  good  reason  for  getting  matters  right 
the  first  time.  The  reward  is  rapid  progress  through  the  lesson.  Thus,  a test  need  not  be  elab- 
orate or  exhaustive,  but  merely  relevant  and  sufficiently  difficult  to  make  a guessing  strategy 
by  the  trainee  unattractive.  This  degree  of  monitoring  student  progress  is  similar  to  that  asso- 
ciated with  conventional  schooling,  in  contrast  for  example  to  the  more-intensive  diagnostic 
methods  occasionally  realized'by  live  tutors  and  by  advanced  computer  systems. 

Another  major  need  is  to  help  the  author  organize  his  lesson  material.  The  approach  here 
is  to  let  the  SME  choose  the  most  appropriate  lesson  type  from  among  several  models,  and  to 
provide  an  authoring  procedure  for  each  model.  The  two  designs  emphasized  to  date  are  the 
Pure  Procedure  and  the  TPI,  which  are  explained  below.  They  are  organized  around  the  work 
experience,  either  natural  or  contrived  sequences  of  tasks  prototypical  of  what  the  trainee  is 
likely  to  encounter.  An  advantage  of  basing  lessons  on  the  work  sequence  is  that  it  is  more  fa- 
miliar than  other  modes  of  lesson  organization  such  as  concepts  or  principles. 

In  summary,  the  need  to  impart  an  adequate  understanding  of  equipment  and  systems  does 
not  appear  to  depend  on  sophisticated  training  methods.  The  goal  of  OJT  is  to  produce  a tech- 
nician, not  an  engineer.  Organizing  lessons  around  the  work  process  is  a "natural"  for  the 
author  - as  well  as  for  the  trainee,  work  manager,  and  other  SMEs.  It  serves  the  overall  goal 
of  improving  work  performance  by  delivering  instruction  in  work  procedures  in  a way  that  can 
be  generalized  to  a variety  of  tasks  on  a given  equipment  or  system. 
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II!.  AUTHORING  ACTIN'  [TH  IS 


Two  examples  of  authoring  activities  are  described  here.  Kach  was  conducted  under  con- 
trolled conditions  to  establish  further  the  feasibility  of  the  approach. 

A.  Pure  Procedure  as  Model  Lesson 

The  Pure  Procedure  presents  the  step-by-step  actions  required  to  execute  a job  on  an  actual 
piece  of  equipment  in  sufficient  detail  for  a novice  technician  to  follow.  In  maintenance,  the  in- 
formational  requirements  are  the  same  as  for  the  JPA  (Job  Performance  Aid).  Displays  show 
explicitly  the  location  of  parts,  terms  are  operationally  defined,  and  standards  of  safety  of  the 
trainee  and  the  integrity  of  the  equipment  are  maintained.  The  overall  goal  of  Pure  Procedure 
in  the  computer-based  instructional  environment  differs,  however,  from  the  JPA,  which  is  pri- 
marily a means  to  effect  equipment  repair  with  unsophisticated  teelinicians. 

following  a procedure  serves  also  as  a kind  of  familiarization  training.  The  user  learns 
facts  about  the  equipment  and  the  content  of  routine  maintenance  operations.  At  regular  inter- 
vals, the  work  performance  is  subject  to  checks.  The  trainee  enters  information  about  results 
of  his  operations,  such  as  meter  readings,  and  the  instructional  terminal  compares  these  with 
expected  values.  It  branches  to  remedial  actions  whenever  there  is  an  erroneous  result.  The 
purpose  of  these  checks  is  not  only  to  correct  errors  by  feedback,  but,  at  a broader  level,  to 
keep  the  user  constantly  accountable  for  his  work.  In  the  l.TS-5  delivery  unit,  his  inputs  are 
also  recorded  on  a tape  cassette.  Both  immediate  checks  and  automated  logging  assure  quality 
of  work  performance  and  enhance  learning  about  the  equipment  and  the  maintenance  process. 

f actual  material  learned  in  this  manner  is,  of  course,  transient  in  memory.  Also,  it  does 
not  provide  understanding  that  allows  transfer  to  new  situations,  e.g.,  meeting  contingencies 
not  provided  for  in  the  procedure.  It  is  higlily  useful  in  that  it  prepares  the  trainee  to  do  stan- 
dardized procedures  and,  after  a few  repetitions,  enables  him  to  do  so  with  the  aid  of  a con- 
densed JPA,  a work  card.  More  significantly,  it  provides  a sound  basis  for  a continuation  of 
training  in  a task-oriented  environment.  About  12  hr  of  this  kind  of  material,  standard  main- 
tenance procedures  on  the  AN/TRC-97 A radio  set,  were  prepared  by  Lincoln  Laboratory.  These 
have  been  in  use  for  1 year  at  the  5th  Combat  Communications  Group,  the  project  field  site  at 
Robins  A1B,  Georgia.  Novice  personnel,  roughly  40  in  all,  who  were  unfamiliar  with  this 
equipment  have  received  their  initial  training  on  these  Pure  Procedural  lessons.  Formal  evi- 
dence is  lacking,  but  the  maintenance  section  manager  continues  to  rely  on  this  capability  be- 
cause he  feels  it  enhances  work  performance  and  benefits  training. 

An  opportunity  for  Lincoln  personnel  to  get  a closer  look  at  the  Pure  Procedure  authoring 
process  was  afforded  when  SSgt  Case  of  the  2nd  CCG  was  sent  to  Lincoln  on  temporary  assign- 
ment for  2 weeks  in  February  of  this  year.  Mis  mission  was  to  be  trained  to  maintain  a new 
I.TS-5  being  installed  at  his  home  station,  and  to  learn  about  the  software  and  courseware  tech- 
nology of  the  machine.  As  part  of  his  indoctrination,  he  produced  a 1-hr  Pure  Procedure  on 
X-l  Diode  Alignment  of  the  I.TS-5.  First,  he  received  training  on  the  I.TS-5  hardware;  then, 
he  converted  an  abbreviated  SOP  extant  on  paper  to  an  illustrated,  step-by-step  procedure.  His 
effort  was  monitored  closely,  and  thus  served  as  a test  of  the  lesson  for  authors  on  development 
of  this  kind  of  lesson. 

Development  of  a Pure  Procedure  requires:  (1)  accurate  initial  specification  of  content, 

(2)  step-wise  guidance  in  lesson  preparation,  (3)  review  of  intermediate  lesson  products  by  an 
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TABLE  1 

TWO  EXAMPLES  OF 

PREPARATION  OF  LESSON  FINAL 

DRAFT 

Lesson  Type 

Pure  Procedure 

TPI 

Author 

SSgt  Case 

Sr  A Robinson 

Title 

Alignment  of  X-Y  Diodes.  . . 

Voice  Circuits  of  AN/TRC-97A 

Lesson  Duration  (hr) 

1 

i 

Number  of  Frames 

23 

45 

Author 

Other 

SME 

LTS-5 

Training 

Coordinator 

Author 

LTS-5 

Other  Training 

SME  Coordinator 

Preparation  Time: 

Initial  Specification 

0 

10 

0 

25 

5 

10 

Prepare  Draft  | 

18 

1 

4 

40 

0 

8 

Review  and  Revise 

2 



1 

4 

12 

5 

2 

c 

L 

In- TUSS’l 

EXPLANATION 

EXPLANATION 

EVALUATION 

i— CORRECTION 

Flowchart  of  TIM  model  lesson. 


I TS  training  expert,  and  (4)  review  of  content  by  other  SMEs.  In  this  instance,  authoring  went 
smoothly  and  the  amounts  of  time  spent  in  producing  an  initial  draft  are  shown  in  Table  L Tests 
on  a few  trial  subjects  uncovered  only  a few  minor  flaws  in  this  product.  As  is  often  the  case 
in  the  development  of  a Pure  Procedure,  a change  of  content  is  a more  likely  basis  for  revision 
than  form  — a work  procedure  must  be  accurate  in  every  detail.  SSgt  C ase  was  a novice  author 
and  was  relatively  unsophisticated  in  the  content  subject  matter,  I.TS-5  hardware  technology. 

Yet  the  lesson  development  process  functioned  well,  allowing  the  author,  with  assistance  from 
other  SMKs  and  a member  of  the  staff  acting  as  the  training  coordinator,  to  produce  a final  prod- 
uct efficiently. 

B.  The  TI’I  as  a Model  Lesson 

The  other  kind  of  lesson  under  development  is  the  TPI  (Task  Procedure  and  Interpretation). 

It  is  a lesson  model  that  has  appeared  sparingly  in  the  past,  perhaps  because  to  be  effective  it 
depends  heavily  on  the  capability  of  a computer  to  monitor  progress,  and  computers  have  not  as 
vet  been  very  practical  for  application  at  the  work  site.  The  advent  of  the  I.TS-5,  combining 
microprocessor  capability  with  audio/visual  displays,  makes  it  reasonable  to  deliver  instruction 
in  this  mode  for  OJT. 

The  formal  structure  of  a sub-unit  of  TPI  instruction  is  shown  in  Fig.  1.  There  are  two 
sequential  phases  within  a sub-unit;  explanation,  information  about  part  of  a system  or  equip- 
ment; and  task,  a work  instruction  supported  by  the  prior  explanation.  Within  each  phase,  the 
primary  information  is  supported  by  optional  information,  by  performance  tests,  and  by  reme- 
dial information. 

In  the  explanation  phase,  an  option  to  get  Help  provides  additional  detail  to  assist  those 
trainees  that  do  not  fully  grasp  the  Explanation.  The  presence  of  a Help  option  at  each  step  is 
a means  to  accommodate  both  novices  and  experienced  personnel  within  the  same  lesson.  The 
Evaluation  is  a test  of  learning.  It  serves  (1)  as  a partial  check  of  the  trainee's  capability  to 
use  the  information  in  solution  of  a problem,  (2)  as  a timely  reminder  of  the  relevance  of  the 
concepts,  and  (3)  as  an  inducement  to  learn.  The  Correction  attempts  to  assist  the  trainee  who 
fails  the  test;  it  is  keyed  when  necessary  to  the  kind  of  error  made.  When  the  Evaluation  is 
failed  repeatedly,  the  trainee  may  be  advised  to  "Go  find  your  instructor." 

The  task  phase  is  entered  after  there  is  a reasonable  assurance  that  the  explanation  has 
been  mastered.  Analogous  functions  are  served  in  this  phase;  the  trainee  is  performing  a task 
and  selects  a course  of  action  consistent  with  the  prior  Explanation.  Typically,  the  Help  option 
explains  sub-tasks  in  more  detail,  such  as  how  to  set  up  a measurement.  Evaluation  is  based 
on  the  measurement  or  other  result  inserted  by  the  trainee.  Correction  usually  indicates  how 
the  outcome  was  unacceptable  in  terms  of  the  previous  explanation;  again,  the  trainee  who  fails 
repeatedly  or  in  a particularly  unfortunate  manner  is  directed  to  obtain  assistance.  The  trainee 
moves  through  a sequence  of  units  of  this  kind  that  he  or  she  is  likely  to  experience  in  a typical 
work  situation. 

The  model  of  the  TIM  in  Fig.  1 is  presented  to  the  SME  author  as  a guide,  but  is  rarely  real- 
ized in  exactly  this  form.  Only  the  essential  blocks  are  included.  Some  functions  may  be  elab- 
orated in  several  frames  or  even  in  a sub-unit;  inessential  form  and  content  are  omitted.  The 
task  phase  may  even  precede  the  explanatory  phase. 

In  planning  a TIM,  the  first  step  is  to  select  a typical  work  process  (experience)  or  a stan- 
dard procedure  for  the  lesson  model.  Characteristically,  a TIM  serves  the  purpose  of  showing 
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the  trainees  how  component  tasks  meet  system  requirements  and  how  various  conditions  affect 
the  choice  of  actions.  It  is  sometimes  profitable  not  to  require  the  trainee  to  carry  out  opera- 
tions on  real  systems,  even  if  otherwise  feasible,  but  to  give  a simulated  work  experience  free 
of  irrelevant  procedural  details.  One  way  or  another,  the  TI’I  is  suited  for  capturing  the  kind 
of  subject  matter  expertise,  job-related  practical  judgment,  and  reasoning  capabilities  that  are 
essential  to  the  operation  and  maintenance  of  complex  systems  and  are  particularly  difficult  and 
costly  to  impart  in  the  OJT  environment. 

(.  I example  of  TI’I  Preparation 

The  project  operates  within  the  "ISD/l/TS-5"  training  activity  of  TAC  COMM  AH  I'.  A of  the 
Air  Force  Communications  Service.  A TI’I  lesson  was  prepared  on  field  installation  of  audio 
circuits  of  the  AN/TRC-97A.  The  requirements  for  this  lesson  were  established  by  the  5th  CCG 
under  a mission  to  design  new  training  materials,  using  the  principles  of  ISD  (Instructional  Sys- 
tem Design)  as  presented  in  AFM  50-2.  It  was  implemented  by  SrA  Robinson,  an  SME  on  tem- 
porary assignment  to  the  ISD/l.TS-5  team. 

At  the  initiative  of  the  maintenance  managers,  two  areas  of  training  were  identified  as  par- 
ticularly critical  by  the  Wideband  Section  of  the  5th  CCG.  Robinson  was  assigned  to  develop  one 
of  them  — the  lesson  on  "handling  requests  and  reports  from  the  Tech  Controller  by  the  opera- 
tion/maintenance technician  of  the  AN/TRC-97A."  lie  first  compiled  a list  of  33  kinds  of  re- 
quests and  reports,  and  listed  appropriate  actions  for  each.  This  product  was  reviewed  by  his 
S3V1F  peers  and  amended.  At  this  point,  Robinson  was  ready  to  begin  to  prepare  a lesson  and 
was  temporarily  assigned  to  l.incoln  l.aboratory  to  do  the  job. 

Initial  review  of  the  requirements,  with  a project  staff  member  acting  as  an  LTS-5  training 
coordinator,  centered  around  the  content  and  goals  of  the  lesson.  The  lesson  is  a simulation  of 
a typical  operational  sequence  and  assumes  that  the  trainee  is  familiar  with  basic  techniques  of 
patching  and  measurement.  The  topic  was  narrowed  to  setting  up  the  voice  section  of  the  multi- 
plexer. The  procedures  involved  did  not  cover  all  possible  situations,  but  provided  an  opportu- 
nity to  explain  most  of  the  concepts  of  the  equipment  and  of  operational  doctrine  required  to  make 
a proper  working  installation  of  the  set.  An  overall  lesson  plan  was  devised  that  followed  a typ- 
j'  > 'ario  in  establishing  the  voice  circuits  when  a TRC-97A  is  first  set  up  in  the  field  on 
mobile  deployment.  Fach  episode  became  a a .b-unit  of  instruction  within  the  TI’I,  as  described 
in  the  previous  section.  It  is  intended  to  be  used  with  the  actual  equipment  on  hand. 

The  author  then  proceeded  to  follow  the  TI’I  lesson  design  procedure  through  the  prescribed 
stages:  preparation  of  outline,  flowchart,  and  draft  frames  in  notebook  form.  Each  stage  was 

presented  to  the  author  by  the  training  coordinator;  the  author  then  worked  independently,  and 
the  product  was  reviewed  and  revised  before  proceeding  to  the  next  stage.  The  final  draft  note- 
book was  reviewed  by  two  peer  SMEs  and  was  run  manually  on  two  novices  at  the  5th  CCG.  It  is 
being  readied  on  fiche  to  be  tested  at  the  2nd  CCG  at  I’atrick  AKB  (see  Sec.  D below). 

The  author  was  able  to  infer  principles  of  lesson  design  from  examples,  and  to  differentiate 
essential  from  nonessential  information  for  the  target  training  population.  The  procedure  that 
underlies  the  Ti’I  development  process  seemed  to  assist  the  author  greatly  in  orderly  develop- 
ment of  the  lesson,  in  communication  with  the  training  coordinator,  and  in  critical  evaluation  of 
his  products.  However,  preparation  of  any  kind  of  individualized  material  is  a time-consuming 
task.  Times  in  preparation  of  a final  draft  (ready  for  conversion  to  film  cards)  are  shown  in 
Table  I.  The  author  spent  77  hr,  and  nad  30  hr  of  the  peer  and  training  coordinator  support  in 
the  preparation  of  the  lesson  which  will  provide  about  1 hr  of  training. 
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Four  problems  the  technician  faces  in  mobile  deployment,  that  were  identified  as  important 
in  the  specifications,  were  not  handled  in  this  particular  TPI  lesson;  so  each  was  dealt  with  in 
a second,  follow-on  lesson  made  up  of  four  parts.  A work  problem  was  presented  and  carried 
through  several  stages  to  a final  resolution.  However,  the  usual  explanatory  phase  ahead  of  each 
task  phase  w as  eliminated.  This  lesson  was  planned  as  a follow'-up  since  it  was  judged  that  the 
requisite  explanatory  information  was  adequately  covered  in  the  original  TI’L  Whenever  an  im- 
proper action  was  selected,  interpretive  corrections  were  given  as  in  F ig.  Z.  Such  a lesson 
might  be  called  an  Kxercise  1 esson  — it  resembles  the  exercises  at  the  end  of  chapters  in  text, 
but  goes  beyond  them  in  providing  remedial  information,  both  conceptual  and  practical  in  nature. 


1 ig.  Z.  Flowchart  of  Kxercise  Lesson 
developed  for  set  up  of  voice  circuits 
for  AN/TKC-97A. 
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* AN  EXAMPU  OF  A TROU8U  IS  THAT  TECH  CONTROL  AT  "A"  REPORTS 
THAT  "D"  IS  NOT  GETTING  HIS  TONE  ON  CHANNEL  6. 

TEACH  STEP  INVOLVES  THE  IDENTIFICATION  OF  WHAT  TO  PROBE 
IN  THE  SYSTEM,  OR  HOW  TO  PROBE,  OR  THE  EVALUATION  Of 
A RESULT. 

An  Kxercise  I.esson  may  have  a number  of  goals  — to  test  comprehension,  to  consolidate 
learning,  to  cover  special  cases,  and  so  forth.  In  brief,  the  TPI  presents  the  prototypical  ex- 
perience and  the  Kxercise  I.esson  deals  with  the  exceptional  and  miscellaneous  cases.  A num- 
ber of  Kxercise  Lessons  have  been  developed  previously  for  the  LTS  in  basic  electronics,  dig- 
ital systems  engineering,  and  mining  technology.  Robinson,  having  already  prepared  a Tl’I, 
required  very  little  additional  instruction  in  the  preparation  of  his  Kxercise  Lesson.  In  view  of 
this  finding,  there  is  little  reason  to  provide  a separate  procedure  for  authors  to  support  this 
design. 

U.  Test  Plans 

TAC  COMM  ARKA  of  AF't'S  has  provided  an  opportunity  to  conduct  operations  at  an  addi- 
tional site,  the  Znd  CCG  at  Patrick  AFB.  This  Group  has  manpower,  equipment,  and  other  re- 
sources available  to  conduct  authoring  activities  and  to  evaluate  lesson  products  in  formal  trials. 
An  LTS-5-based  training  operation  of  the  AN/TRC-97A  has  been  established  at  that  site,  and  an 
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extensive  test  of  the  new  lesson  is  planned  as  an  initial  effort.  This  test  will  involve  10  to  20 
novice  technicians  who  have  undergone  familiarization  training  on  the  l.TS-S.  The  lesson  will 
lie  administered  to  each  individual,  and  pre-  and  post-tests  of  skill  knowledge  and  work  perfor- 
mance will  be  used  to  measure  learning.  Mach  performance  test  measures  the  ability  to  cope 
with  typical  problems,  as  simulated  on  the  LTS.  Although  it  would  be  preferable  to  assess  the 
transfer  of  training  to  actual  work  performance  in  field  operations,  the  feasibility  of  obtaining 
"hard"  data  of  this  kind  is  not  yet  apparent.  This  test  is  a prototype  of  a series  of  tests  to  val- 
idate the  concepts  and  techniques  of  the  proceduralized  approach  to  authoring  by  SMKs. 
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